Self-supervised Regularization for Text Classification

Abstract Text classification is a widely studied problem and has broad applications. In many real-world problems, the number of texts for training models limited, which renders these prone to overfitting. To address this problem, we propose SSL-Reg, data-dependent regularization approach based on self-supervised learning (SSL). SSL (Devlin et al., 2019a) an unsupervised that defines auxiliary tasks input data without using any human-provided labels learns representations by solving tasks. supervised task are performed simultaneously. The unsupervised, defined purely human- provided labels. Training model can prevent from being overfitted limited class in task. Experiments 17 text datasets demonstrate effectiveness our proposed method. Code available at https://github.com/UCSD-AI4H/SSReg.